自我介紹

陳紹慶
慈濟大學人類發展與心理學系專任副教授

開放科學中心大使
心理科學加速器創始成員；訓練委員會助理監督
scchen.com

我正在進行的研究主題

閱讀句子的心像模擬歷程

Orientation Stanfield & Zwaan (2001)

retrieved from Bergen (2012)

Shape Zwaan, Stanfield, & Yaxley (2002)

retrieved from Bergen (2012)

Color Connell(2005, 2007)

retrieved from Bergen (2012)

潛在應用：人類語言的圖像式處理

Q1：你認為這樣的研究渉及捏造(fabrication)與篡改(falsification)嗎？

Diederik Stapel造假事件

Daryl Bem超感知覺研究爭議

Source: https://youtu.be/42QuXLucH3Q

Source: https://youtu.be/42QuXLucH3Q

Criticisms

JPSP rejected failed replication(Ritchie, Wiseman, & French, 2012)

HARKing (Hypothesizing After the Results are Known; Kerr, 1998)

End of Science?

QRP檢核表

Six ways to p-hacking

1. Stop collecting data once p <.05.
   
2. Analyze many measures, but report only those with p <.05.
   
3. Collect and analyze many conditions, but only report those with p < .05.
4. Use covariates to get p < .05.
5. Exclude participants to get p < .05.
6. Transform the data to get p < .05.

Nelson(2014)

推薦閱讀

Q1:操作有疑問的研究都是涉及捏造與篡改嗎？

Q2:經不起考驗的研究結果會傷害研究誠信嗎？

預先註冊(Preregistration):第一階段沒有同儕評審

註冊報告(Registered Report):第一階段有同儕評審

註冊再現研究(Registered Replication Report)

(Lindsay, Simons, & Lilienfeld, 2016)

預先註冊(Preregistration)

The researcher creates as detailed a description of his or her plans for a study as possible and saves those plans in a time-stamped, uneditable archive. This record can be shared with reviewers, editors, and other researchers.

My understanding: Preregistration and registered report in social science follow the framework of Randomized Controlled Trial but make a space for researcher flexibility.

註冊你的專案

Open-Ended Registration

OSF-Standard Pre-Data Collection Registration

AsPredicated registration

Prereg Challenge

• Check the template.
  
• Encourage researchers conduct exploratory study in preregisration form.
  
• Submit your preregistered study to the eligible journal.
  
• Publish your study before Dec. 31, 2018. COS reward you US$ 1000.

How to select my preregistration?

Pilot: Study without sample size justification; Secondary data set
EXP: Study with sample size justification

註冊報告(Registered Report)

AIMS Neuroscience submission pipeline

(Chambers, Feredoes, Muthukumaraswamy, & Etchells, 2014)

註冊再現研究(Registered Replication Report)

Daniel Simons on Pre-registration (from “Improving your statistical inferences”)

When publication bias joined:

APS RRR

• Psychological science should emphasize findings that are robust, replicable, and generalizable.

• Direct replications are necessary to estimate the true size of an effect.

• Well-designed replication studies should be published regardless of the size of the effect or statistical significance of the result.

Ego depletion

Facial Feedback Effect

Q3:我該不該改進研究操作？

Q4:我要從什麼地方開始改進？

降級版權力姿勢效應

Good theory, Bad evidence

• Failed replications (Ranehill, 2015)
• Publication bias (Carney, Cuddy, & Yap, 2015; Simmons & Simonsohn, 2017)
• Sloopy methods: p-hacking(Cherry picking) and HARKing (Bailey, LaFrance, & Dovidio, 2017; Cesario, Jonas, & Carney, 2017; Gronau et al., 2017; Jonas et al., 2017; Klaschinski, Schnabel, & Schröder-Abé, 2017; Latu, Duffy, Pardal, & Alger, 2017; Ronay, Tybur, van Huijstee, & Morssinkhof, 2016)
• Till present, there is a reproducible but slight feeling of power.

自砸招牌的飲食心理學大師

Wansink boasted how to make paper by QRPs

(Original date: 2016/11/21)

GRIM: A tool to check the spooky roundings.

James Heathers	Nick Brown	Jordan Anaya	Tim van der Zee

All 4 Wansink-boasted papers have QRPs:

Questions in Smater Lunchrooms

• The fundamental paper had been retracted because they collected the data from kinder gardens, not the elementary schools they claimed in this paper.

Brian Wansink is on his position till this talk.

• 改變發表／補助機制
• 資料品質鑑識工具
• 開放研究資料累積平台

Codes of Conduct for Good Academic Practice

TU/E ERB statement:

Evaluation tools of data quality

Curate Science

1. Transparency
2. Analytic reproducibility
3. Analytic robustness
4. Effect replicability

(LeBel, vanpaemel, McCarthy, Earp, & Elson, 2017)

Ego depletion (Recommended browser: Firefox)

如何分享你的資料與分析程序

• Open access to your analytic codes in your preregistration form.
  
• Share your data and materials in the open repository.
• Request the supports of journal editors, funders, and institutions.

Q5:公開資料對我有何好處？

Q6:要如何公開資料才能提昇研究品質？

協作模式

• 臨時結盟
• 投案平台
• CERN model

Many Labs project

Klein et al (2014)

CRSP special issue(2017): power posing

Feeling of power: all partiipants	Feeling of power: unfamiliar only

Quentin et al. (2017)

Code of Conduct

The Society for the Improvement of Psychological Science (SIPS) holds professional conferences and other events to enable its members to work to improve methods and practices in psychological science, build professional networks, and receive continuing education.

SIPS events

• Workshop
• Hachthon
• Unconference
• Lightning Talks
• Meetup
• Mentor/Mentee lunch

Psychological Science Accelerator

After you signed in PSA …

• Collecting data
• Reviewing study submissions
• Serving on one of our operational or advisory committees

• Providing feedback on the procedures, policies, and governance of the Accelerator

• Check PSA introductory paper

• Five operations and five advisory Committees
• There are/will be five associate directors (one for two committees) and 20 assistant directors (two for one committees).
• PSA will open for the submission of project proposal for each 6 months. Each wave will accept 2~3 projects.

Workflow of PSA projects

How PSA manage your project

• Project proposal has to be registered report.
• If project PI is single, the committees will assign one collobrator help compose the well-written proposal to be submitted.
• Lab PI decide if he/she will join this project at this time.

• Each participating lab takes the research ethic approval respectively.

• If I have more time, I’d like to share more about PSA and colloborative projects.

提昇與維持研究品質是建立誠信的基礎

自由提問